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FOREWORD 


Within  today’s  Army,  two  conflicting  forces  are  at  work;  decreasing  resources  for 
training  and  increasing  demands  for  highly  trained  and  proficient  personnel.  Force  reductions  and 
other  cost  efficiencies  require  the  Army  to  reduce  its  expenditures  for  high-fidelity  institutional 
and  field  training.  At  the  same  time,  introduction  of  more  complex  systems  and  equipment, 
changes  to  doctrine  and  organization,  and  a  changing  geopolitical  landscape  require  that  training 
be  more  committed  to  quality  and  efficiency  than  ever  before. 

To  meet  these  challenges.  Congress  provided  Fiscal  Year  1994  research  and  development 
funding  for  the  establishment  of  the  Force  XXI  Training  Program  (formerly  known  as  the  Virtual 
Brigade  Training  Program)  at  Fort  Knox,  KY.  The  intent  of  this  program  is  to  explore  and  utilize 
simulation  technologies  and  instructional  principles  to  create  structured  training  programs  that 
fully  leverage  available  resources  in  providing  efficient,  effective  training  to  brigade  staffs.  The 
focus  is  on  both  preparing  and  equipping  the  Army  of  the  21'*  century  and  ensuring  that  today’s 
Army  is  sufficiently  ready  to  provide  the  foundation  for  continuing  change  and  modernization. 

The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI),  Armored 
Forces  Research  Unit  (AFRU)  at  Fort  Knox;  the  Force  XXI  Training  Program;  and  the  U.S. 

Army  Armor  Center  (USAARMC)  joined  forces  to  sponsor  training  research  and  development 
(R&D)  for  one  elem'ent  of  the  Congressionally-mandated  effort;  simulation-based  training  for  the 
conventional  mounted  brigade.  The  R&D  work  was  performed  under  a  project  known  as  the 
Combined  Arms  Operations  at  Brigade  Level,  Realistically  Achieved  Through  Simidation 
(COBRAS).  The  project  is  an  element  of  Research  Task  2228  -  R03. 


This  research  report  describes  a  methodology  for  developing  structured  simulation-based 
training.  It  provides  the  foundation  and  background  for  the  process  presented  in  the  Guide  to 
Development  of  Structured  Simulation-Based  Training  (submitted  to  ARI  for  publication).  The 
guide  incorporates  experience  and  lessons  learned  during  the  COBRAS  work,  and  includes  more 
detailed  discussion  of  training  support  packages  and  formative  evaluation  requirements.  This 
report  and  the  guide  itself  are  designed  for  use  by  training  designers  and  developers,  as  well  as  by 
training  program  reviewers  and  proponents. 


Director 


EDGAR  M.' JOHNSON 
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REPORT  ON  THE  EXPANDED  METHODOLOGY  FOR  DEVELOPMENT  OF 
STRUCTURED  SIMULATION-BASED  TRAINING  PROGRAMS 

EXECUTIVE  SUMMARY 


Research  Requirement: 

The  Combined  Arms  Operations  at  Brigade  Level,  Realistically  Achieved  Through 
Simulation  (COBRAS)  project  was  required  to  design,  develop,  implement,  formatively  evaluate, 
and  document  a  structured,  simulation-based  training  program.  The  program  addressed  staff 
skills  for  selected  members  of  the  staff  of  the  conventional  mounted  brigade.  Two  levels  of 
training  were  required;  a  large-scale  integrated  exercise  that  brought  together  virtually  the  entire 
brigade  combat  team  and  its  assets,  and  a  set  of  more  focused  small-group  vignette  exercises. 

The  programs  were  to  emphasize  brigade  staff  processes  in  all  phases  of  brigade  mission 
operations:  planning,  preparation,  and  execution  (to  include  the  sustainment  efforts  of 
consolidation  and  reorganization).  The  program  documentation  requirement  included  preparation 
of  a  report  on  the  methodology  used  to  develop  the  training. 

Procedure: 

The  training  design  and  development  followed  the  guidance  contained  in  the  Methodology 
for  Development  of  Structured  Simulation-Based  Training  (Campbell,  Campbell,  Sanders,  Flynn, 
&  Myers,  1995).  That  methodology  was  based  on  work  performed  in  development  of  execution- 
only  training  for  platoons,  companies,  and  battalions  in  the  Virtual  Training  Program. 

During  the  course  of  COBRAS  design  and  development,  changes  in  the  original 
methodology  were  necessitated  by  the  focus  on  a  different  echelon  for  the  training  audience,  the 
coverage  of  phases  other  than  execution,  and  the  inclusion  of  combat  support  and  combat  service 
support  elements.  These  changes  did  not  invalidate  the  original  methodology,  which  was 
surprisingly  robust  with  respect  to  changes  in  focus.  Developers  and  analysts  documented  areas 
in  which  the  methodology  remained  constant  and  those  areas  that  required  expanded  guidance. 

Findings; 

The  outcome  of  the  documentation  of  the  methodology  modifications  is  this  report  on  the 
expanded  methodology.  It  incorporates  discussion  of  the  experience  and  lessons  learned  during 
COBRAS  development,  reported  as  a  coherent  presentation  of  the  methodology  process  and  its 
conceptual  underpinnings  in  the  Systems  Approach  to  Training.  A  research  product  has  also  been 
generated,  the  Guide  to  Development  of  Structured  Simulation-Based  Training  (Campbell  & 
Deter,  in  preparation).  The  guide  presents  the  enhanced  methodology  as  a  step-by-step  process, 
and  does  not  require  reference  to  the  original  methodology.  Both  this  report  and  the  guide 
contain  discussion  of  training  support  packages  (TSPs)  and  formative  evaluation  discussion  that 
were  not  included  in  the  original  methodology. 


Utilization  of  Findings; 

Both  this  report  and  the  guide  itself  are  addressed  to  training  designers  and  developers,  as 
well  as  to  training  program  reviewers  and  proponents.  The  report  will  provide  information  and 
justification  for  the  processes  described  in  the  guide.  The  guide  contains  the  procedural 
instruction  and  examples  that  will  enable  other  developers  to  construct  sound,  complete  programs 
and  that  will  assist  reviewers  in  evaluating  the  strengths  and  completeness  of  developed  programs. 
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REPORT  ON  THE  EXPANDED  METHODOLOGY  FOR  DEVELOPMENT  OF 
STRUCTURED  SIMULATION-BASED  TRAINING  PROGRAMS 

Introduction 


Purpose 

This  report  on  the  methodology  for  development  of  structured  simulation-based  training 
describes  the  conceptual  underpinnings  of  the  methodology,  from  its  initial  codification  in  1995  to 
the  current  revision  (the  guide  augmented  by  this  report:  Campbell,  Deter,  &  Quinkert,  in 
preparation).  With  this  discussion  in  hand,  evaluators  and  prospective  users  of  the  process  can 
understand  and  appreciate  (or  take  issue  with)  the  basic  tenets  of  the  methodology,  and  adapt  the 
process  for  their  own  needs. 

The  purpose  of  the  guide  itself  is  twofold: 

•  It  expands  the  original  methodology  by  incorporating  lessons  learned  about  development 
of  structured  simulation-based  training. 

•  It  includes  discussion  pertinent  to  the  development  and  review  of  training  support 
packages  (TSPs). 

The  guide  is  intended  for  use  by  developers,  reviewers,  evaluators,  and  managers  of 
structured  training  programs  and  their  development. 

Background 

In  1993-1995,  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
(ARI)  sponsored  a  research  and  development  effort  to  construct  structured  simulation-based 
training  exercises  for  multiple  echelons  of  the  armored  force.  The  finished  program  is  known  as 
the  Virtual  Training  Program  (VTP^ ).  In  addition  to  the  training  support  package  materials, 
project  personnel  also  produced  a  report  &aT\X\Qd  Methodology  for  Development  of  Structured 
Simulation-Based  Tra/wwg- (Campbell,  Campbell,  Sanders,  Flynn,  &  Myers,  1995).  The 
methodology  is  a  step-by-step  procedure  that  is  based  on  the  work  performed  in  designing  and 
developing  the  VTP  training  materials  and  scenarios.^ 


'  The  program  was  originally  known  as  the  Reserve  Component  (RC)  Virtual  Training  Program  (RCVTP).  As 
active  units  and  schoolhouse  training  managers  became  interested  in  using  the  program,  the  “RC”  designator  was 
dropped.  The  work  was  performed  as  an  ARI  project  entitled  Simulation-Based  Training  Multiechelon  Training 
Program  for  Armor  Units  (SIMUTA),  reported  in  Hoflhnan,  Graves,  Roger,  Fljmn,  and  Sever,  1995.  Additional 
VTP  work  was  performed  in  the  project  Expansion  of  the  Virtual  Training  Program  (SJMUTA-B),  reported  in 
Graves  &  Myers,  in  preparation. 

^  The  methodology  itself  is  not  specific  to  any  particular  technology,  type  of  unit,  unit  level,  or  mission.  It  is, 
however,  most  applicable  to  the  development  of  collective  military  training  exercises  that  focus  on  tactical  skills  in 
planning  and  execution. 
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The  VTP  was  developed  for  the  use  of  U.S.  Army  National  Guard  (ARNG)  units,  to 
provide  them  with  high-quality,  time-compressed  structured  training  in  virtual  and  constructive 
environments.^  Both  offensive  and  defensive  exercises  are  included  for  the  battalion  and 
battalion  staff  (armor  battalion  and  battalion  task  force),  company-level  (armor  company, 
company  team,  and  cavalry  troop),  and  platoon-level  (armor  platoon,  mechanized  infantry 
platoon,  and  scout  platoon).  For  the  staff,  exercises  either  Janus  (constructive  simulation)  as  the 
behind-the-scenes  driver;  all  other  exercises  are  implemented  using  Simulation  Networking 
(SIMNET)  technology  (virtual  simulation). 

Subsequent  to  its  use  in  developing  the  VTP,  the  methodology  (Campbell  et  ah,  1995) 
has  been  used  in  the  construction  of  similar  structured,  simulation-based  training  programs  for 
brigade  staffs'*,  for  units  equipped  with  digitized  weapon  systems^  and  for  conventional 
(nondigitized)  forces  on  different  simulations®.  Adaptations  were  made  as  necessary  to 
customize  the  methodology  for  each  usage.  The  basic  methodology,  however,  has  proven  to  be 
robust  with  respect  to  differences  in  environments  and  training  audience. 

The  most  recently  completed  of  these  programs,  and  the  one  that  serves  as  the  basis  for 
this  report,  is  the  Force  XXI  Training  Program:  Combined  Arms  Operations  at  Brigade  Level, 
Realistically  Achieved  Through  Simulation,  usually  referred  to  as  COBRAS.  On  this  ARI  project, 
training  exercises  were  developed  for  selected  members  of  a  conventional  heavy  mounted 
brigade.  The  training  focuses  on  three  missions  (movement  to  contact,  area  defense,  and 
deliberate  attack)  and  on  all  phases  of  those  missions  (plan,  prepare,  and  execute,  including 
consolidation  and  reorganization). 

Two  types  of  training  package  were  developed:  the  COBRAS  Brigade  Staff  Exercise 
(BSE)  and  the  Brigade  Staff  Vignettes.^  In  brief,  the  two  types  of  training  are  described  as 
follows: 


^  “Constructive  simulations”  are  those  where  the  technology  represents  actions  and  consequences,  but  does  not 
replicate  the  full  physical  environment.  “Virtual  simulation”  are  those  in  which  users  actually  interact  with  a 
replication  of  the  physical  environment,  in  much  the  same  way  that  they  would  with  the  real  environment.  “Live 
simulation”  refers  to  those  situations  where  the  real  environment  is  reproduced  in  a  less  than  full  mode.  Whether  or 
not  either  constructive  or  virtual  simulation  technologies  is  used,  all  performance  practice  training  involves  some 
degree  of  live  simulation,  in  that  it  involves  real  people  in  less-than-real  conditions. 

Simulation-Based  Mounted  Brigade  Training  Program  (SIMBART),  reported  in  Roger,  Long,  Britt,  Sanders, 
Broadwater,  and  Brewer,  1996. 

’  Simulation-Based  Training  for  Digitized  Units  (SIMUTA-D),  reported  in  Winsch,  Garth,  Ainslie,  and  Castleberry, 
1996. 

Structured  Training  for  Units  in  the  Close  Combat  Tactical  Trainer  (STRUCCTT),  an  ongoing  ARI  project. 

’’  The  COBRAS  exercises  and  TSPs  and  the  development  process  are  more  fully  described  in  the  project  report: 
Combined  Arms  Operations  at  Brigade  Level,  Realistically  Achieved  Through  Simulation  (COBRAS)  Project: 
Development  and  Lessons  Learned  (Graves  et  al.,  in  preparation). 
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•  The  BSE  integrates  a  large  segment  of  the  brigade  staff  and  staffs  of  subordinate  and 
supporting  units,  in  a  scenario  based  on  a  continuous  storyline  of  all  of  the  missions  and 
phases.  It  makes  use  of  the  Brigade/Battalion  Battle  Simulation  (BBS). 

•  The  vignettes  are  a  series  of  13  independent  training  exercises  designed  for  small 
groups  of  brigade  staff  members.  Each  vignette  focuses  on  a  well-defined  and 
delimited  event,  in  order  to  provide  concentrated  practice  on  specific  skills  and 
behaviors  for  a  small  group  training  audience. 

From  the  time  the  VTP  was  first  being  developed  until  the  COBRAS  training  programs 
were  delivered,  there  has  been  increasing  interest  in  the  training  support  packages  (TSPs)  that 
operationally  define  structured  training  programs.  TSPs  have  been  variously  defined: 

•  In  the  VTP-derived  methodology,  the  TSP  was  envisioned  as  a  two-part  set  of 
materials:  those  for  the  trainer,  and  those  for  the  trainee. 

•  Several  months  after  the  methodology  was  published,  in  an  early  COBRAS  briefing  for 
MG  Maggart  (Commanding  General,  Armor  Center  and  Fort  Knox),  there  were  four 
components:  tactical  materials,  training  audience  materials,  trainer  materials,  and 
simulation  materials. 

•  Currently,  The  U.  S.  Army  Training  and  Doctrine  Command  (TRADOC)  Regulation 
350-70  calls  for  five  components:  tactical,  training  audience,  trainer,  simulation,  and 
administrative  (Department  of  the  Army  [DA],  1995). 

While  these  ways  of  classifying  materials  vary,  the  content  itself  is  substantively  the 
same  for  all  three  models.  Increased  attention  to  the  content  of  TSPs,  as  well  as  their  organizing 
architecture,  led  to  more  definitive  descriptions  of  model  TSPs.  The  current  guide  includes 
specific  considerations  for  developing  TSP  components  for  structured  training  programs. 

Contents  and  Organization  of  the  Report  and  Guide 

The  report  itself  comprises  four  sections: 

•  This  section  of  the  report.  Introduction,  contains  a  discussion  of  the  background  of  the 
methodology  and  characteristics  of  structured  training  and  TSPs. 

•  The  next  section  of  the  report.  Overview  of  the  Methodology,  contains  a  discussion  of 
the  methodology  approach,  delineating  its  relationship  to  the  Systems  Approach  to 
Training  [SAT]  (DA,  1988)  and  describing  the  formative  evaluation  philosophy  for  the 
methodology. 

•  The  third  section  of  the  report.  Overview  of  the  Guide,  contains  an  overview  of  the 
organization  of  the  guide. 

•  The  fourth  section  of  the  report  is  the  Summary,  and  includes  discussion  of  the  potential 
users  of  the  report  and  the  guide. 
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This  report  is  intended  to  provide  details  concerning  the  development  of  the 
methodology.  It  can  be  read  and  understood  without  reference  to  the  guide.  Likewise,  the  guide 
itself  may  be  read  and  used  without  reading  the  report. 

The  guide  itself  contains  detailed  discussion  and  instructions  on  the  use  of  the 
methodology,  not  specific  to  any  simulation  technology,  mission,  or  unit  type.  The  guide: 

•  describes  the  activities  to  be  followed  in  developing  the  exercise  components  (e.g., 
scenarios  and  trainer  materials), 

•  indicates  where  revision  loops  occur, 

•  details  review  and  tryout  (i.e.,  formative  evaluation)  requirements,  and 

•  describes  the  products  from  each  activity  and  how  they  feed  into  subsequent  activities. 

This  version  of  the  guide  both  expands  and  revises  the  earlier  version  (Campbell  et  al., 
1995).  Although  the  original  version  held  up  robustly  during  subsequent  uses,  it  was,  in 
retrospect,  somewhat  restrictive  in  scope.  As  experience  with  construction  of  structured  training 
has  accumulated,  understanding  of  the  opportunities  for  flexibility  has  increased.  This  version  of 
the  methodology  seeks  to  encourage  creativity,  offer  more  flexible  guidance,  and  allow  for 
departures  from  the  model. 

While  the  tenets  for  development  of  structured  training  are  appropriate  in  nonmilitary 
training  settings  and  in  a  live  simulation  mode,  this  methodology  is  couched  in  military  and 
constructive/virtual/live  simulation-based  training  terms.  As  appropriate,  guidance  is  provided 
for  development  of  training  for  both  tactical  battlefield-oriented  training  and  also  for  training  of 
skills  such  as  planning  and  decision-making  processes. 

The  appendix  in  this  report  contains  a  job  aid  version  of  the  methodology,  in  the  form  of 
an  outline  and  job  aid  checklist  for  developers  and  reviewers.  While  the  guide  contains  details 
and  discussion  of  the  procedure,  the  appendix  summarizes  the  process  and  serves  as  a  handy 
reference  for  more  experienced  developers. 

Structured  Training  Characteristics 

The  term  “structured  training”  is  \videly  used  in  current  training  discussions.  It  is 
generally  assumed  that  there  exists  a  commonly  accepted  definition.  However,  the  term  is  rarely 
defined  at  the  start  of  the  discussion,  despite  the  fact  that  various  definitions  are  in  use.  Some  of 
those  definitions  are  as  follows: 

•  Structured  training  is  often  said  to  be  task-driven,  rather  than  event-driven.  This  may 

mean  that 

o  practice  and  feedback  are  directed  at  specific  tasks  rather  than  at  the  events 
that  happen  to  occur  during  an  exercise;  or 
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o  training  should  be  planned  to  address  task  proficiency  needs,  rather  than  to 
satisfy  a  scheduled  training  event  (such  as  a  rotation  to  a  combat  training  center 
[CTC]  or  a  field  training  exercise  [FTX]). 

•  Another  usage  of  the  term  “structured  training”  implies  that  a  carefully  designed  set  of 
initial  conditions  (e.g.,  mission,  enemy,  troops,  terrain,  and  time  available  [METT-T]) 
has  been  defined  for  the  exercise.  Once  the  exercise  begins,  little  additional  structure  is 
imposed. 

•  Yet  another  definition  refers  to  the  “. . .  totality  of  training  development,  training 
support,  and  guidance  for  effective  application/use  within  the  unit  or  institution” 
(Brown,  1993,  p.  2). 

None  of  the  usages  or  definitions  above  restricts  the  focus  of  the  training  to  a  particular 
audience,  setting,  or  implementation  model.  In  fact,  the  term  has  been  used  to  refer  to  various 
training  programs,  including  the  following: 

•  Training  that  is  directed  to  the  individual,  small  groups,  or  units  (military  or  otherwise). 

•  Training  for  a  single  level  or  echelon,  or  training  for  several  levels  linked  by  means  of 
common  scenarios  or  through  simultaneous  involvement  in  the  training. 

•  Training  that  involves  use  of  constructive  or  virtual  simulations  or  training  that 
involves  only  live  simulation  (although  all  training  involves  a  degree  of  live 
simulation). 

•  Training  that  is  turn-key  training  for  the  participants  as  well  as  training  that  requires 
participant  planning  and  preparation. 

In  all  cases,  however,  the  task-directed  structure  is  intended  to  maximize  the  training  value  for  all 
participants. 

A  comprehensive  definition  of  structured  training  is  presented  in  Figure  1 .  It 
encompasses  each  of  the  definitions  and  usages  described  above,  and  forms  the  basis  of  the 
methodology  presented  in  the  guide.  It  indicates  the  importance  of  a  careful  process  of  needs 
analysis  and  critical  task  selection  as  the  basis  for  the  training  program.  These  preliminary  steps 
lead  into  preparation  of  the  supporting  materials  that  comprise  the  TSP.  The  TSP  provides  two 
kinds  of  tools: 

•  The  scenario  and  support  guides  that  provide  the  cues  to  control  the  training  process 
(i.e.,  the  behaviors  and  activities  of  the  trainers  in  controlling  the  flow  of  the  exercise), 
and 

•  The  training  objectives,  observation  guides,  and  associated  after  action  review  (AAR) 
materials  that  support  the  learning  process  (i.e.,  the  activities  of  the  training  audience 
and  the  means  for  providing  feedback  to  them). 
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STRUCTURED  TRAINING 


For  the  programs  developed  under  ARI  at  Fort  Knox,  structured  training  is  operationally 
defined  as  training  that  is  deliberately  and  purposefully  constructed  so  as  to  focus  on  specific 
training  objectives.  The  concept  incorporates  several  key  features: 

•  a  focus  on  performance  of  selected  critical  tasks 

•  standardized  exercise  control  to  ensure  practice  of  the  tasks 

•  standardized  feedback  to  correct  and  reinforce  performance  on  the  selected  tasks 

•  exercise  support  by  means  of  comprehensive  training  materials. 

The  development  methodology  described  here  incorporates  those  features  while 
following  accepted  instructional  design.  It  also  incorporates  two  other  features  that  are  not 
essential  to  the  definition  of  structured  training: 

•  It  takes  advantage  of  virtual,  constructive,  and/or  live  simulation  capabilities. 

•  It  is  embedded  in  the  context  of  tactically  realistic  scenarios,  causing  the  unit  to  be 
immersed  in  the  (simulated)  tactical  situation  that  elicits  performance  of  the  critical 
tasks. 
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The  training  focus  is  ensured  by  means  of  careful  attention  to  standardized 
implementation  and  doctrinal  guidance,  and  is  accomplished  by  adhering  to  the  following 
guidelines*: 

•  The  conditions  that  provide  the  setting  for  the  training  should  be  closely  defined  and 
crafted  to  be  realistic  for  task  performance. 

•  The  selected  tasks  that  form  the  training  objectives  should  be  doctrinally  correct.  This 
means  that,  whether  they  are  drawn  directly  from  published  documentation  or  are 
elaboration  of  documented  procedures,  there  should  be  acceptance  of  the  technical 
accuracy  of  the  tasks  and  how  they  must  be  performed. 

•  The  training  should  take  advantage  of  and  work  within  the  capabilities  of  the  selected 
simulator/simulation. 

•  The  training  materials  should  support  all  aspects  of  the  training;  that  is,  the  unit's  focus 
should  be  on  participating  in  the  training,  rather  than  on  designing  and  developing  the 
training,  or  on  observing  and  evaluating  their  own  performance. 

•  The  materials  should  permit  standardized  (that  is,  replicable)  implementation  of  the 
program,  so  that  the  selected  tasks  are  cued  consistently. 

•  The  training  should  support  a  selected  training  sequence  with  regard  to  task 
performance  (e.g.,  crawl-walk-run,  natural  order,  hierarchical  order). 

•  The  training  should  make  use  of  performance  observers,  and  materials  should  direct 
their  attention  to  the  key  training  objectives. 

•  The  training  should  be  designed  to  fit  within  the  typical  unit's  time  and  resource 
constraints. 

•  The  training  materials  should  be  flexible,  so  that  different  situations  can  be 
accommodated,  and  should  also  include  warnings  and  explanations  about  what  should 
not  be  modified. 

Training  Support  Package  Products 

The  products  of  a  structured  simulation-based  training  development  project  comprise  the 
TSP.  For  the  ARI-sponsored  projects  and  as  defined  in  TRADOC  Reg  350-70  (DA,  1995),  the 
TSP  structure  is  shown  in  Figure  2.  The  TSP  components  include: 

•  Tactical  materials  for  the  training  audience  and  other  participants.  Such  materials 
may  include  operations  orders  (OPORDs)  and  other  mission-specific  materials  to  be 
used  in  unit  preparation  and  rehearsals,  prepared  messages  and  scripted  materials  that 
are  used  during  the  exercise  to  cue  tasks,  and  descriptions  of  personnel  and  equipment 
status  to  set  the  scenario  conditions. 


*  The  originally  proposed  principles  have  been  modified,  based  on  experience  and  lessons  learned  in  subsequent 
projects. 
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THE  TRAINING  SUPPORT  PACKAGE 


Tactical  materials:  I — 

•  OPORDs  and  other  mission-specific  materials 

•  Prepared  messages  and  scripted  materials 

•  Descriptions  of  personnel  and  equipment  status 


Unit  preparation  materials: 

•  Guidance  on  exercise  selection 

•  Guidance  on  unit  preparation 

•  Description  of  time,  personnel,  and  facility  requirements 


^nex  C,  dPORD  95-53 
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Guidance  for  other  participants: 

•  Ob^rver  materials  on  task  performance  and  feedback  sessions 

•  Instructions  for  operation  of  the  simulation  components 

•  Guidance  for  roleplayers 


Administrative  guidance  for  managers : 

•  Instructions  for  setting  up  the  training 

•  Instructions  for  controlling  scenario  events 


Simulation  tapes  and  documentation: 

•  Initialize  the  system 

•  Provide  starting  conditions 

•  Show  graphic  control  measures 
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Figure  2.  Components  of  a  training  support  package  for  a  structured  simulation-based  training 
program. 


•  Preparation  materials  for  the  training  audience.  These  assist  unit  leaders  in  selecting 
the  appropriate  training  for  their  needs,  provide  guidance  on  unit  preparation,  and 
describe  the  time,  personnel,  and  facility  requirements. 

•  Guidance  for  other  participants.  This  includes  observer  materials  that  direct  attention 
to  specific  task  performance,  describe  acceptable  performance,  and  outline  the  feedback 
sessions;  instructions  for  operation  of  the  simulation  components;  and  guidance  for 
roleplayers  who  interact  with  the  imit  during  the  exercise.  Because  all  of  these 
participants  who  are  not  the  primary  training  audience  are  actually  “trainers,”  these 
guides  represent  the  “train  the  trainer”  materials. 

•  Administrative  guidance  for  managers  of  the  training.  These  include  instructions  and 
suggestions  for  setting  up  the  training  and  controlling  scenario  events. 
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•  Simulation  tapes  and  documentation.  These  are  used  with  the  selected  constructive  or 
virtual  simulation  to  initialize  the  system,  provide  starting  conditions  (e.g.,  locations, 
task  organization),  and  show  graphic  control  measures.  The  documentation  is  provided 
as  back-up  in  case  the  initialization  tapes  are  unusable. 

This  outline  for  the  components  of  a  TSP  allows  developers  considerable  latitude  in 
organizing  and  presenting  their  materials.  Such  flexibility  is  essential,  because  every  structured 
training  program  will  require  an  individually  prepared  TSP,  specific  to  the  requirements  of  that 
program. 

Structured  training  programs  and  TSPs  are  and  should  be  closely  associated.  In  fact, 
those  who  work  on  the  ARI  training  development  projects  have  come  to  regard  the  two  concepts 
as  inseparable.  Even  a  structured  program  that  appears  to  have  no  TSP  will  be  seen  to  have  an 
implicit  TSP,  in  the  mind  and  person  of  a  proficient  trainer.  On  the  other  hand,  a  TSP  can  be 
developed  for  an  unstructured  training  program,  but  that  TSP  will  be  incomplete  in  that  it  will 
not  include  task  cues.  Therefore,  throughout  this  methodology  report  and  the  guide,  structured 
training  and  the  TSP  that  goes  with  it  are  equally  the  focus. 

Note  on  the  Original  Methodology  Report 

The  original  methodology  (Campbell  et  al.,  1995)  has  certain  items  that  may  be  of 
interest  to  developers  and  that  are  not  included  in  this  revision: 

•  Section  4,  Training  is  addressed  to  developers  who  will  have  the  assistance  of  a 
dedicated  training  cadre  when  the  program  is  implemented. 

•  Section  5,  Using  the  Methodology  for  Training  Program  Modifications,  discusses 
various  topics  related  to  the  development  methodology,  including  extension  of  existing 
exercises  to  other  METT-T  and  description  of  ongoing  efforts  using  the  methodology. 

•  Several  different  worksheets  were  designed  to  aid  in  preparation  and  quality  control  of 
structured  training  programs;  fill-in-the-blank  worksheets  are  provided  in  Appendix  B 
of  the  original  methodology  (Campbell  et  al.,  1995) 

•  Appendix  C  of  the  original  methodology  (Campbell  et  al.,  1995)  contains  discussion 
and  examples  of  how  the  activities  were  performed  in  developing  the  VTP.  The 
process  for  each  echelon  is  discussed  in  those  sections,  as  follows: 

o  Part  \,  Platoon-Level  Training 

o  Part  2,  Company-Level  Training 

o  Part  3,  Battalion-Level  Training 

o  Part  4,  Battalion  Staff  Training. 


9 


Overview  Of  The  Methodology 


This  section  establishes  the  methodology’s  relationship  to  the  existing  SAT  model  (DA, 
1988)  and  describes  the  developmental  and  formative  evaluation  activities  in  the  methodology. 

The  Methodology  and  the  Systems  Approach  to  Trainine 

The  methodology  for  developing  structured  simulation-based  training  exercises 
originated  in  the  SAT  model  (DA,  1988).  The  SAT  process  comprises  five  phases: 

•  Analysis — Determine  the  training  requirement,  training  audience,  and  appropriate 
training  media. 

•  Design — Prepare  the  specifications  of  the  training  coiriponents. 

•  Development — Construct  and  refine  the  various  training  package  components. 

•  Implementation — ^Deliver  the  training  in  an  operational  setting. 

•  Evaluation-— Assess  success  in  achieving  the  program's  training  goals. 

The  development  methodology  described  here  parallels  the  SAT  process  in  most  respects. 
However,  the  SAT  activities  phases  have  been  narrowed  in  this  methodology  to  focus  on  the 
development  of  structured  training  that  uses  simulation.  This  methodology  concentrates 
primarily  on  Analysis  (including  certain  initial  decisions  and  selection  of  training  objectives). 
Design  (including  other  initial  decisions  and  specification  of  the  exercise  limits),  and 
Development  (construction  of  TSP  components,  tryouts,  and  formative  evaluation). 

Developmental  Activities  in  the  Methodology 

There  are  four  phases  to  this  development  methodology,  as  shown  in  Figure  4: 

•  Phase  1:  Initial  Decisions — Determine  the  training  requirement  (e.g.,  mission  and 
enemy  type,  terrain,  time  constraints,  number  of  exercise  start  points,  difficulty  level), 
training  audience  (e.g.,  unit  type  or  echelon,  personnel  within  unit),  and  appropriate 
training  environment  (i.e.,  specific  simulator/simulation). 

•  Phase  2:  Select  Training  Objectives — ^Focus  the  training  on  critical  tasks  and 
performance  standards  in  support  of  the  training  requirements,  and  ensure  that  those 
tasks  can  be  performed  in  the  selected  simulator/simulation  environment. 

•  Phase  3:  Design  Scenario  and  Exercise  Structure — Determine  the  limits  of  each 
exercise  with  reference  to  METT-T;  generate  the  tactical  framework  for  the  exercises; 
specify  the  events  within  each  exercise;  define  and  put  substance  to  the  roles  to  be 
played  by  the  simulation  and  by  various  types  of  trainers;  and  crosswalk  the  training 
objectives  to  events  within  the  exercises. 
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PHASES  OF  THE  DEVELOPMENT  METHODOLOGY 


FORMATIVE 

EVALUATION 


Phase  1 : 
Initial  Decisions 


Phase  4: 

Develop  Training 
Support  Package  (TSP) 


Decisions  on: 

•  Target  audience 

•  Training  context 

•  Simulation  technology 


Phase  2: 

Designate  Training 
Objectives 


Identify  task  sources  and  tasks 
Refine  task  list  for  simulation  support 
Select  tasks  that  support  the  mission 


Phase  3: 

Design  Scenario  and 
Exercise  Outline 


•  Design  exercise  scenario 

•  Prepare  exercise  context  and 
specifications 

•  Outline  events  and  build  exercise 


Design  TSP  structure 
Prepare  TSP  materials 


Figure  3.  The  four  phases  in  the  methodology  for  development  of  structured  simulation-based 
training. 


•  Phase  4:  Develop  Training  Support  Package  (TSP) — Construct  and  try  out  all  of  the 
written  and  simulator/simulation-based  components  of  the  training  program,  including 
materials  for  the  trainers  and  for  the  participating  unit. 

These  four  phases  need  not  be  done  in  a  strictly  linear  fashion.  There  should  be 
considerable  overlap  among  the  phases.  For  example,  understanding  of  the  initial  decisions  will 
change  as  more  is  learned  about  the  simulation  and  the  tasks;  information  that  feeds  into 
development  of  the  TSP  will  be  captured  during  documentation  of  initial  decisions;  and  so  on. 

The  SAT  implementation  and  evaluation  phases  are  not  explicitly  replicated  in  the 
development  methodology.  Both  operational  implementation  and  the  summative  evaluation  take 
place  after  the  activities  covered  by  the  methodology,  when  the  training  package  is  received  by 
the  user.  However,  the  methodology  does  contain  the  developmental  parallel  processes:  pilot 
and  trial  implementations  and  formative  evaluations. 
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Formative  Evaluation  Activities  in  the  Methodology 


The  development  methodology  incorporates  a  series  of  required  and  critical  formative 
evaluation  steps  throughout  the  development  process.®  Some  of  these  are  formal  activities  (such 
as  map  exercises  conducted  by  the  design  team,  tryouts  with  representative  or  surrogate  units  and 
individuals,  technology-driven  tryouts,  or  content  reviews  with  experts  and  stakeholders  or 
proponents'®),  but  formative  evaluation  also  includes  a  continuous  attention  to  the  need  for 
revisions  and  improvements.  The  purpose  of  formative  evaluation,  both  the  formal  activities  and 
the  continuous  revisions,  is  to  insure  the  quality  of  the  product  by  attending  to  quality  throughout 
development.  In  Figure  3,  formative  evaluation  is  shown  as  underlying  all  phases  of 
development.  This  continuous  improvement  and  revision  process,  which  derives  from  the 
formative  evaluation  attention,  is  what  causes  design  and  development  to  be  non-linear,  as 
mentioned  above. 

In  the  guide,  appropriate  formative  evaluation  activities  are  described  for  each  of  the 
developmental  activities.  Each  formative  evaluation  activity  has  a  specific  focus,  appropriate  to 
the  development  activity,  and  the  information  gathered  is  used  to  revise  and  refine  products  or  to 
ensure  the  accuracy  and  usability  of  the  products.  Briefly,  the  formative  evaluation  activities  for 
each  phase  of  the  methodology  include  the  following: 

•  For  Phase  1  (Initial  Decisions),  the  formative  evaluation  task  will  be  to  make  sure  that  all 
of  the  parameters  of  the  proposed  structured  training  have  been  examined  by  those  who 
have  commissioned  the  training.  All  of  the  directives  that  have  been  given  by  the 
training  program  proponent  (the  office  or  individual  who  is  sponsoring  or  has  directed 
that  the  development  be  done)  should  be  documented  by  the  developers  and  approved  by 
that  proponent.  This  will  ensure  that  the  development  is  on  track  with  the  initial  intent. 

It  is  crucial  that  the  proponent  review  and  approve  of  any  changes  in  the  design  and 
development  work. 

•  In  Phase  2  activities  (Designate  Training  Objectives),  developers  will  be  identifying  the 
specific  training  objectives  for  the  program.  These  are  the  tasks  that  the  program  will 
address  and  train,  and  the  standards  for  task  achievement.  The  formative  evaluation  in 
Phase  2  should  take  the  form  of  an  expert  review  of  the  selected  tasks.  This  review 
should  involve  any  offices  or  agencies  who  are  responsible  for  doctrine  and 
documentation  for  the  subject  matter  (e.g.,  doctrine  or  training  or  tactics  directorates). 
These  agencies  have  a  vested  interest  in  the  product  and  a  responsibility  for  its  accuracy; 
they  are  also  knowledgeable  and  can  provide  valuable  information  and  support. 

•  As  development  proceeds  in  Phase  3  (Design  Scenario  and  Exercise  Structure),  the 
formative  evaluation  will  become  more  active,  involving  map  exercises  and  simulation- 
controlled  exercises,  and  finally  exercises  with  representative  (or  surrogate)  participants 
(usually  referred  to  as  “pilots”).  These  exercises  are  intended  to  evaluate  the  technical 

’  The  formative  evaluation  activities  were  termed  “quality  reviews”  in  the  original  methodology. 

By  “proponent,”  we  mean  whoever  decided  that  the  training  was  required.  This  person  or  agency  is  usually  the 
primary  client  for  the  development  effort. 
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content  of  the  development,  with  regard  to  both  doctrine  and  simulation.  There  is  still  a 
requirement  for  proponent  and  expert  reviews  of  the  scenario,  to  ensure  that  the  evolving 
design  is  still  consistent  with  the  intent. 

•  Finally,  in  Phase  4  (Develop  Training  Support  Package),  the  formative  evaluation  will 
involve  representative  individuals  and  units  in  trying  out  the  program's  TSP  (referred  to 
as  “trials”).  By  this  time,  developers  should  have  performed  reviews  to  ensure  that  the 
exercises  are  doctrinally  correct  and  that  they  are  matched  to  simulation  capabilities. 

They  are  now  ready  to  check  on  how  usable  the  exercises  are  and  whether  training  (and 
learning)  occurs.  These  trials  are  extensive  and  require  careful  planning,  rigorous  and 
intensive  information-gathering,  and  documentation  of  resulting  actions.  They  are  the 
final  point  of  revision  and  will  demand  excruciating  attention  to  detail  in  order  to  ensure 
that  all  materials  are  complete  and  correct. 

In  the  VTP  development,  all  of  these  formative  evaluation  steps  were  followed,  and  were 
often  repeated  as  additional  revisions  were  made.  In  the  COBRAS  development,  several 
expedient  methods  had  to  be  developed  because  of  the  inability  to  obtain  sufficient  representative 
participants;  they  are  discussed  in  the  COBRAS  final  report  (Graves  et  ah,  in  preparation). 

Overview  of  the  Guide" 

The  detailed  guidance  regarding  the  application  of  the  methodology  is  contained  in  the 
Guide  to  Development  of  Structured,  Simulation-Based  Training.  There  are  nine  Activities 
within  the  four  phases  of  the  methodology."  In  the  guide,  each  phase  of  the  methodology  is 
introduced  by  a  discussion  of  the  purpose  of  that  phase,  along  with  a  list  of  the  activities  in  the 
phase.  Each  activity  is  then  presented  with  a  lengthy  explication  of  the  considerations  for  that 
activity. 

The  Development  Process 

The  nine  activities  should,  in  general,  be  performed  in  the  order  described.  However,  the 
process  is  not  meant  to  be  rigid.  The  continuous  attention  to  formative  evaluation  during  an 
activity  will  often  cause  you  to  go  back  and  revise  the  products  of  an  earlier  activity.  In  fact, 
developers  should  regard  the  entire  process  as  flexible,  and  be  prepared  to  move  back  and  forth 
between  activities  during  the  development,  revising  decisions  and  products  as  necessary. 

Whenever  decisions  or  products  are  revised,  developers  must  trace  back  through  earlier 
development  and  correct  all  related  products,  whether  they  are  interim  products  or  parts  of  the 
final  TSP.  We  cannot  emphasize  too  strongly  the  importance  of  keeping  all  products  current  and 


"  Extracts  of  this  section  are  also  included  in  foreword  to  The  Guide  'to  the  Methodology  for  Development  of 
Structured  Simulation  Based  Training. 

Ten  activities  were  included  in  the  original  methodology  (Campbell  et  al.,  1995).  Two  of  the  activities  (3.1  and 
3.2)  were  combined  into  a  single  activity  in  this  version. 
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in  agreement  with  each  other.  We  should  also  point  out  that,  in  our  experience,  this  is  one  of  the 
most  difficult  challenges  in  development. 

Although  the  guide  presents  a  succinct  procedure  for  developing  structured  simulation- 
based  training,  the  guide  alone  will  not  be  sufficient  for  development.  Subject  matter  expertise 
for  the  selected  technology,  for  associated  military  aspects  (e.g.,  the  mission  type,  unit  type, 
operations,  and  enemy  tactics),  and  for  training  and  instructional  design  will  be  required 
throughout  the  development  and  review  processes. 

The  Methodology  Outline  (Appendix) 

The  appendix^of  the  guide  (also  included  as  an  appendix  to  this  report)  presents,  in 
outline  form,  the  phases,  activities,  and  formative  evaluation  considerations  in  the  methodology. 
As  appropriate,  some  of  the  most  important  design  and  development  considerations  are  also 
listed.  For  each  of  the  activities,  the  outline  indicates  the  activity’s  product  or  outcome.  Not  all 
of  the  indicated  products  are  used  in  the  final  TSP;  some  are  interim  products  that  assist 
developers  in  moving  through  the  development  process. 

The  appendix  serves  as  a  job  aid  or  checklist  for  the  development  process  and  includes 
little  detail  or  explanation.  It  is  designed  to  help  maintain  high  quality  in  the  TSP  materials.  It 
lists  the  characteristics  of  correct  and  complete  TSP  materials.  Thus,  it  should  be  used  to  remind 
experienced  developers  and  reviewers  of  the  critical  elements  of  the  development  process  and  the 
TSPs. 

Formative  Evaluation  Activities 

The  various  types  of  formative  evaluation  activities  described  in  the  guide  provide 
information  appropriate  to  each  stage  of  development,  from  the  basic  decisions,  through  the 
development  of  supporting  pieces  of  the  exercises,  to  actual  materials  in  the  TSP.  The  formative 
evaluation  activities  will  draw  your  attention  to  the  many  ways  that  the  materials  must  be 
carefully  checked,  verified,  and  corrected. 

In  each  of  the  activity  discussions  in  the  guide,  the  "Formative  Evaluation"  step  is 
emphasized  by  means  of  a  separately  headed  section  to  ensure  that  it  receives  sufficient  attention. 

Summary 

This  report  is  intended  to  accompany  the  Guide  to  Development  of  Structured  Simulation- 
Based  Training.  It  has  presented  the  history  of  ARI-sponsored  structured  training  programs.  It 
also  contains  a  discussion  of  structured  training  and  training  support  packages.  Finally,  the 
report  has  described  the  basic  structure  of  the  methodology  and  the  guide. 


14 


By  following  the  process  described  in  the  guide,  various  objectives  for  various  users  may 

be  met: 

•  Training  developers  will  be  confronted  with  most  of  the  critical  considerations  that 
must  be  faced  in  designing  and  developing  structured  simulation-based  training 
programs. 

•  Reviewers  of  training  program  development  projects  or  of  TSPs  will  find  that  the 
methodology  outlines  the  issues  that  should  be  addressed  in  development  and  the 
considerations  that  should  be  incorporated  into  the  TSP. 

•  Evaluators  of  training  programs  and  TSPs  will  be  able  to  identify  the  probable  causes 
of  weaknesses  in  programs  by  tracing  through  the  critical  development  steps. 

•  Managers  who  oversee  the  development  and  implementation  of  structured  training 
programs  will  be  better  equipped  to  modify  the  developmental  process  and  elements  of 
the  TSP,  through  a  full  appreciation  of  the  prescribed  methodology. 


Performing  Organizations 

The  COBRAS  Team,  which  was  responsible  for  the  design  and  development  of  the 
training  program  that  formed  the  basis  for  this  revision  of  the  methodology,  includes:  Human 
Resources  Research  Organization  (HumRRO);  Hughes  Training,  Incorporated;  BDM,  Federal; 
and  PRC,  Incorporated. 
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APPENDIX 


Guide  to  Development  of 
Structured  Simulation-Based  Training 

OUTLINE  OF  THE  METHODOLOGY 


Phase  1.  Initial  Decisions 


Activity  1 . 


Product  or 
outcome 


Formative 

evaluation 


Decision  areas 
(pages  7-10 
of  the  Guide) 


Document  Initial  Decisions 


Documentation  of  decisions 


Proponent  review 


1.  Target  training  audience 

2.  Training  context 

•  Mission  type 

•  Enemy  type 

•  Terrain/locale 

•  Unit  type 

3.  Simulation  technology 

4.  Other,  e.g.: 

•  Exercise  time 

•  Number  of  entry  points 

•  Nature  of  entry  points 

•  Linkages  to  other  programs 

•  Guidance  for  training  priorities 

•  Trainer  and  other  resources 


Phase  2.  Designate  Training  Objectives 


NOTE 

Either  Activity  2.2  or  Activity  2.3  can  be  done  first.  Do  whichever  you 
think  will  reduce  the  task  list  most. 

W  NOTE 

Phase  2  and  Phase  3  (designing  the  scenario  and  exercise  structure)  will 
probably  be  iterative. 

Activity  2.1  Identify  Task  Sources,  Tasks,  and  Standards 

Product  or 

outcome 

Tasks  and  standards;  task  sources 

Formative 

evaluation 

Internal  and  proponent  review 

Sources 
(pages  12-13 
of  the  Guide) 

•  Doctrinal  —  ARTEP-MTPs,  FMs,  proponent  agency  lists 

•  Other  —  j  ob  and  task  analysis 

Task  detailing 
(page  13 
of  the  Guide) 

•  Task  statement  —  lowest  level  of  collective  behavior  that  has 
accompanying  conditions  and  standards 

•  Conditions  statement  —  description  of  situation,  environment,  and 
initiating  cues  that  should  cause  a  task  to  be  performed 

•  Standard  —  statement  of  correct,  acceptable,  ideal  accomplishment  of 
a  task 

Activity  2.2  Refine  Task  List  for  Simulation  Support 


Product  or  Task  list  annotated  to  show  tasks  that  can  be  fiilly  or  partially  performed  and 
outcome  observed  in  the  simulation 


Formative  Internal  and  proponent  review 

evaluation 


Judgments  of 
simulation 
support 
(pages  14-15 
of  the  Guide) 


Use  a  rule-based  system 

•  Use  more  than  one  judge;  reconcile  judgments 

•  Use  task  expert  and  simulation  expert 

•  Document  partial  trainability  (which  parts) 

•  Determine  whether  the  task: 

•  Can  be  performed  in  simulation 

•  Should  be  trained  in  simulation  setting 

•  Can  be  performed  in  the  simulation  setting,  though  not 
simulation-supported 


Activity  2.3  Select  Tasks  That  Support  the  Mission 


Product  or  Reduced  task  list,  annotated  to  show  the  tasks  (or  parts  of  tasks)  that  will  be 
outcome  performed  and  can  be  observed  in  the  context  of  the  mission,  along  with 

appropriate  conditions  and  standards 


Formative  Proponent  review;  training  and  doctrine  agency  review 

evaluation 


Selecting  tasks 
(and  standards) 
(page  16 
of  the  Guide) 


•  Are  part  of  the  mission(s)  to  be  training  (per  Activity  1) 

•  Can  and  should  be  trained  in  the  simulation,  can  be  observed 

•  “Other”  guidance  for  training  priorities  satisfied 
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Phase  3.  Design  Scenario  and  Exercise  Structure 


NOTE  Phase  2  and  Phase  3  (designing  the  scenario  and  exercise  structure)  will 
probably  be  iterative. 

Phase  3  activities  are  closely  tied  and  highly  interdependent.  Compare  unit’s 
and  higher  echelon’s  missions  and  scenario  outlines  frequently  to  ensure 
tactical  correspondence. 


Task  sequencing 

Common  sequencing  approaches: 

(pages  17-18 
of  the  Guide) 

•  Crawl-walk-run 

•  Natural  order 

•  Hierarchical 

•  Easy-to-difficult 

Activity  3.1  Design  the  Scenario 


Product  or 
outcome 


Draft  of  the  "concept  of  the  operation"  with  sketch  of  graphic  overlay,  draft  of 
unit  OPORD,  or  other  method  of  presentation;  for  training  unit  and  higher 
echelons  as  necessary 


Formative  Map  exercises 

evaluation 


Continued  on  next  page 
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Activity  3.1  Design  the  Scenario,  Continued 


Design 
principles 
(page  19 
of  the  Guide) 


Components  of 
mission  outline 
(page  20 
of  the  Guide) 


1 .  Situation  should  be  realistic 

•  Cues  match  real-world  conditions 

•  Tactical  materials  resemble  actual  materials  (1-2  levels  up) 

•  Plausible  activities  at  higher  echelons 

2.  Enemy  should  be  realistic 

•  Organization,  equipment,  tactics,  techniques,  procedures 
consistent  with  its  doctrine 

3.  Real-time,  real-space  events 

•  Minimal  magic  moves  and  reconstitution 

•  Time  only  moves  forward 

•  Constant  terrain 

•  Partitions  at  natural  breaks 


1 .  Initial  locations  (friendly  and  enemy) 

2.  Major  events  list 

3.  Participants  (actual  and  notional,  friendly  and  enemy) 

4.  Terrain  area  or  environment 

5.  Task  by  event  crosswalk 


Battle-oriented 
exercise 
products 
(page  20 
of  the  Guide) 


•  Description  of  mission  and  commander’s  intent 

•  Enemy  plan  (locations,  objectives,  mission,  intent,  etc.) 

•  Terrain  to  support  mission,  sketch  of  graphic  control  measures 

•  Scenario  timeline  showing  friendly  and  enemy  activities 
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Activity  3.2  Prepare  Exercise  Context  and  Specifications 


Product  or 
outcome 


Formative 

evaluation 


Product 
description 
.  (page  22-23 
of  the  Guide) 


Context,  specifications,  and  execution  details  for  exercises 


Simulation-controlled  exercise  to  verify  the  following: 

•  Match  between  specifications  and  simulation 

•  Suitability  of  terrain 

•  Exercise  length 

•  Correlation  of  forces 


1.  Narrative/graphic  representation 

2.  Friendly  and  enemy  situations 

•  Equipment/personnel  status 

•  Recent  events 

•  Starting  locations  (approximate) 

3 .  Unit  specifications  at  starting  point 

•  Unit  identifiers  and  type  unit,  how  represented 

•  Initial  status  of  each  system 

4.  Exercise  ending  point — approximate  location  or  event 

5.  Scenario  description — exercise  intent,  i.e.,  overall  objective,  general 
statement  of  conditions 

6.  Tasks/training  objectives 
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Activity  3.3  Outline  Events  and  Build  Exercise 


Product  or 
outcome 


Formative 

evaluation 

(page  25-26  of 
the  Guide) 


Elements 
(page  24 
of  the  Guide) 


Master  event  list  showing  cues,  expected  performance,  and  critical  tasks  or 
subtasks;  simulation  files 


Pilot  with  knowledgeable  personnel  to  check  the  following: 

•  Scenario  and  order  are  appropriate  for  terrain  and  events 

•  Exercise  specifications  for  all  units  are  correct 

•  Locations  permit  exercise  to  proceed  as  planned 

•  Event  cues  cause  tasks  to  happen 

•  Performance  is  observable 

•  Performance  standards  are  clear  and  observable 


•  Cue  to  start  event 

•  Unit  response 

•  Task  to  be  observed 
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Phase  4.  Develop  Training  Support  Package  (TSP) 


NOTE  TSPs  are  unique  to  the  training  programs  they  support. 


Necessary  inputs 


•  What  will  the  implementation  setting  be? 

•  Who  are  the  users  (implementers  and  training  imit)? 

•  How  will  they  use  the  materials? 

•  What  should  the  materials  help  them  do? 


Activity  4.1  Design  TSP  Structure 


Product  or 
outcome 


Design  outline  for  TSP  components  that  considers  (at  a  minimum): 

•  Use  of  tactical  and  other  scenario  materials 

•  Unit  preparation  (training  audience) 

•  Materials  to  aid  or  train  other  participants 

•  Simulation  needs 

•  Exercise  management 

•  TSP  packaging 


Formative  Proponent  review  and  review  by  representative  users 

evaluation 


Continued  on  next  page 
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Activity  4.1  Design  TSP  Structure,  continued 


Considerations 
(expanded  list) 
(pages  28-29 
of  the  Guide) 


1 .  Tactical  and  other  scenario  materials 

•  What  is  needed,  when  is  it  distributed,  who  gets  it,  how  do  they 
know  what  to  do  with  it? 

2.  Unit  preparation  (training  audience) 

•  Prior  experience,  current  skill  level,  time  available  for 
preparation,  what  preparation  is  needed,  what  media  for  providing 
information? 

3.  Materials  to  aid  or  train  other  participants 

•  Who  are  the  other  participants  (observers,  roleplayers, 
interactors),  prior  experience,  functions  or  roles,  what  job  aids  can 
be  used? 

4.  Simulation  needs 

•  How  will  scenario  be  tr2insferred  to  simulation,  who  does  it,  what 
instructions  needed? 

5.  Exercise  management 

•  Who  will  manage  the  training,  prior  experience,  resources? 

6.  TSP  packaging 

•  How  will  the  TSP  be  packaged  and  distributed? 

•  Be  aware  of  differences  between  the  shelf  (master)  copy  and 
distribution  copy. 
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Activity  4.2  Prepare  TSP  Materials 


Product  or 
outcome 


Formative 

evaluation 


TSP  principles 
(page  38 
of  the  Guide) 


Expert  review 
guidelines 
(page  39  of  the 
Guide) 


The  TSP,  as  outlined  in  Activity  4. 1 . 


Expert  reviews,  trials  with  representative  personnel 


•  Clear  organization 

•  Easily  readable  and  understandable  appearance 

•  Timely  use  of  graphics  and  tables 

•  Extensive  use  of  appropriate  job  aids 


•  Simulation  experts  for  materials  used  to  program  or  configure 
simulation 

•  Doctrinal  experts  for  tactical  materials  and  AAR  materials 

•  Someone  familiar  with  training  needs  and  constraints  for  advance 
materials  (unit  preparation)  and  management  materials 

•  Instructional  design  experts  for  structure  and  presentation  mode 

•  Someone  for  roleplay  of  unpacking,  reproducing,  distributing 


Continued  on  next  page 
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Activity  4.2  Prepare  TSP  Materials,  Continued 


Trial  guidelines 
(page  39-40 
of  the  Guide) 


1 .  Use  actual  (representative)  participants  in  all  positions,  if  possible 

2.  Use  actual  TSP  materials 

3.  Let  participants  use  materials;  intervene  only  in  case  of  fatal  errors 

4.  Collect  information  via 

•  Observations 

•  Group  discussions  before,  during,  after  exercise 

•  Structured  interviews 

•  Questionnaires 

5.  Focus  on 

•  Assumptions  and  expectations 

•  Methods  for  conducting  exercise 

•  Clarity  and  utility  of  materials 

•  Completeness  of  materials  and  absence  of  extraneous  materials 

6.  Have  a  formal  plan 
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